Goto

Collaborating Authors

 ap 25









MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation

arXiv.org Artificial Intelligence

While significant progress has been made in single-view 3D human pose estimation, multi-view 3D human pose estimation remains challenging, particularly in terms of generalizing to new camera configurations. Existing attention-based transformers often struggle to accurately model the spatial arrangement of keypoints, especially in occluded scenarios. Additionally, they tend to overfit specific camera arrangements and visual scenes from training data, resulting in substantial performance drops in new settings. In this study, we introduce a novel Multi-View State Space Modeling framework, named MV-SSM, for robustly estimating 3D human keypoints. We explicitly model the joint spatial sequence at two distinct levels: the feature level from multi-view images and the person keypoint level. We propose a Projective State Space (PSS) block to learn a generalized representation of joint spatial arrangements using state space modeling. Moreover, we modify Mamba's traditional scanning into an effective Grid Token-guided Bidirectional Scanning (GTBS), which is integral to the PSS block. Multiple experiments demonstrate that MV-SSM achieves strong generalization, outperforming state-of-the-art methods: +10.8 on AP25 (+24%) on the challenging three-camera setting in CMU Panoptic, +7.0 on AP25 (+13%) on varying camera arrangements, and +15.3 PCP (+38%) on Campus A1 in cross-dataset evaluations. Project Website: https://aviralchharia.github.io/MV-SSM


Supplementary Material for " Learning Superpoint Graph Cut for 3D Instance Segmentation " Le Hui

Neural Information Processing Systems

This supplementary material provides more details on network architecture, visualization, and ablation study of our method. We also analyze the limitation and discuss the impact of our method. D, we discuss the limitations and impacts of our method. The produced feature dimension of 3D U-Net is 32. Finally, we can obtain 32-dimensional super-point features.


Direct Multi-view Multi-person 3D Pose Estimation (Supplementary Material) Tao Wang

Neural Information Processing Systems

Figure S1: (a) Illustration of the proposed hierarchical query embedding and the input-dependent query adaptation schemes. It consist of a self-attention, a projective attention and a feed-forward network (FFN) with residual connections. Fig. S1 (a) illustrates our proposed hierarchical query The decoder of MvP transformer consists of multiple decoder layers for regressing 3D joint locations progressively. Fig. S1 (b) demonstrates the detailed architecture of a decoder layer, Results are shown in Table S1. Table S1: Results of replacing camera ray directions with 2D coordinates in RayConv.Positional Input AP We further investigate the effectiveness of the proposed projective attention by comparing it with the dense dot product attention, i.e., conducting Results are given in Table S2.